Skip to content

fix: make Node microbenches more stable#376

Merged
cheb0 merged 5 commits intomainfrom
0-fix-agg-benchmarks-variance
Mar 18, 2026
Merged

fix: make Node microbenches more stable#376
cheb0 merged 5 commits intomainfrom
0-fix-agg-benchmarks-variance

Conversation

@cheb0
Copy link
Member

@cheb0 cheb0 commented Mar 6, 2026

Description

Use const seed and call ResetTimer. They false report degradation too often, but it's possible to slow down on them actually.

  • I have read and followed all requirements in CONTRIBUTING.md;
  • I used LLM/AI assistance to make this pull request;

@codecov-commenter
Copy link

codecov-commenter commented Mar 6, 2026 •
edited
Loading

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.62%. Comparing base (822b05b) to head (72d5fcc).

Additional details and impacted files 
@@            Coverage Diff             @@
##             main     #376      +/-   ##
==========================================
+ Coverage   71.61%   71.62%   +0.01%     
==========================================
  Files         206      206              
  Lines       15057    15057              
==========================================
+ Hits        10783    10785       +2     
+ Misses       3488     3487       -1     
+ Partials      786      785       -1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 6, 2026

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table
Name Previous Current Ratio Verdict
AggDeep/size=1000-4 ad0347 96051b
4737.00 ns/op 5498.00 ns/op 1.16 🔴
AggDeep/size=10000-4 ad0347 96051b
47818.00 ns/op 56106.00 ns/op 1.17 🔴
AggWide/size=1000-4 ad0347 96051b
4780.00 ns/op 5920.00 ns/op 1.24 🔴
FindSequence_Random/medium-4 ad0347 96051b
12292.95 MB/s 11074.57 MB/s 0.90 🔴

@github-actions
Copy link
Contributor

github-actions bot commented Mar 11, 2026

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table
Name Previous Current Ratio Verdict
AggDeep/size=10000-4 03387c 7bc28e
45145.00 ns/op 55798.00 ns/op 1.24 🔴
And/size=1000-4 03387c 7bc28e
4.29 ns/op 4.82 ns/op 1.12 🔴

@eguguchkin eguguchkin added this to the v0.69.0 milestone Mar 16, 2026
@eguguchkin eguguchkin requested review from dkharms, eguguchkin and forshev and removed request for eguguchkin March 16, 2026 10:09
eguguchkin
eguguchkin reviewed Mar 16, 2026
View reviewed changes
@github-actions
Copy link
Contributor

github-actions bot commented Mar 17, 2026

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table
Name Previous Current Ratio Verdict
AggDeep/size=10000-4 03387c 349c14
45145.00 ns/op 56829.00 ns/op 1.26 🔴
AggWide/size=1000-4 03387c 349c14
5522.00 ns/op 6196.00 ns/op 1.12 🔴
FindSequence_Random/small-4 03387c 349c14
6427.67 MB/s 5371.19 MB/s 0.84 🔴
40.85 ns/op 47.66 ns/op 1.17 🔴
MutexListAppend-4 03387c 349c14
187.29 MB/s 169.75 MB/s 0.91 🔴
ProcessDocuments-4 03387c 349c14
29061.00 B/op 32403.00 B/op 1.11 🔴
101.50 MB/s 92.16 MB/s 0.91 🔴

@github-actions
Copy link
Contributor

github-actions bot commented Mar 17, 2026

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table
Name Previous Current Ratio Verdict
AggDeep/size=10000-4 03387c 8bd3d8
45145.00 ns/op 57069.00 ns/op 1.26 🔴
AggWide/size=1000-4 03387c 8bd3d8
5522.00 ns/op 6223.00 ns/op 1.13 🔴
FindSequence_Random/large-4 03387c 8bd3d8
14232.04 MB/s 12657.88 MB/s 0.89 🔴
FindSequence_Random/medium-4 03387c 8bd3d8
11010.51 MB/s 9913.98 MB/s 0.90 🔴
FindSequence_Random/small-4 03387c 8bd3d8
6427.67 MB/s 5421.49 MB/s 0.84 🔴
40.85 ns/op 47.22 ns/op 1.16 🔴
ProcessDocuments-4 03387c 8bd3d8
29061.00 B/op 32768.00 B/op 1.13 🔴
101.50 MB/s 91.66 MB/s 0.90 🔴

@cheb0 cheb0 merged commit cb93e48 into main Mar 18, 2026
8 checks passed
@cheb0 cheb0 deleted the 0-fix-agg-benchmarks-variance branch March 18, 2026 05:57
@github-actions
Copy link
Contributor

github-actions bot commented Mar 18, 2026

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table
Name Previous Current Ratio Verdict
AggDeep/size=1000-4 822b05 0ba74c
4697.00 ns/op 5669.00 ns/op 1.21 🔴
AggWide/size=1000-4 822b05 0ba74c
5508.00 ns/op 6200.00 ns/op 1.13 🔴
AggWide/size=1000000-4 822b05 0ba74c
507.00 B/op 565.00 B/op 1.11 🔴
5295660.00 ns/op 5902387.00 ns/op 1.11 🔴
FindSequence_Random/small-4 822b05 0ba74c
5436.45 MB/s 4833.82 MB/s 0.89 🔴

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@forshev forshev forshev approved these changes

@eguguchkin eguguchkin eguguchkin approved these changes

@dkharms dkharms Awaiting requested review from dkharms

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

v0.69.0

Development

Successfully merging this pull request may close these issues.

4 participants

@cheb0 @codecov-commenter @forshev @eguguchkin